Compressed video content is often subject to channel impairment during transmission. In some applications, corrupted video bitstreams may reach the video decoder. When this happens, the quality of delivered video can degrade severely if no proper action is taken. Error concealment is a technique often used by the video decoder in this case. Error concealment recovers the lost or damaged video information by utilizing the remaining redundancy inside a compressed bitstream.
Slice loss error concealment has been considered and implemented in the prior art. However, at least one slice has to be received for each lost frame in order to apply the error concealment. Therefore, it is not applicable when a video frame is lost entirely. However, in many applications, especially those operating at low bit-rates, such as 3G wireless networks, a coded frame is often encapsulated into a transmit unit. When such a transmit unit is lost during transmission, a corresponding frame is lost. Slice loss error concealment fails in these applications.
There are some existing simple error concealment algorithms that can handle entire frame losses. For example, one technique is “Frame Copy” (FC), which is a scheme that simply performs pixel-by-pixel copy from a given reference frame to reconstruct a lost frame. Frame Copy is effective for stationary video sequences which have little motion. However, for those video sequences which possess active motion, Frame Copy often produces low PSNR values as well as noticeable error propagation.
“Motion Copy” (MC) is another simple error concealment which can conceal a lost frame. Motion Copy copies the motion field of a given reference frame to a lost frame and then performs motion compensation to reconstruct the lost frame. Motion Copy is effective for video sequences which possess homogeneous motion fields, such as those occurring in camera panning. However, Motion Copy can generate concealment artifacts when there is abundant chaotic motion in a video sequence.
Some more sophisticated error concealment algorithms have been proposed to conceal a lost frame. These error concealment algorithms can be generally divided into block-level and pixel-level algorithms. One such pixel-level error concealment algorithm extrapolates the motion vectors (MVs) at each pixel from the last received frame. Then, for each pixel in the current missing frame, the error concealment algorithm estimates its MV by either averaging the MVs of all overlapping macroblock (MBs) or duplicating it from the MV of the same location in the previous frame.
In another such pixel-level error concealment algorithm, motion estimation based on optical flow was proposed. Estimated motion vectors (MVs) are projected onto a missing frame after spatial regularization to avoid excessive discontinuities. Unfilled pixels in the lost frame are then interpolated spatially.
All of the above algorithms operate at pixel-level, which estimate the MV for each pixel and then perform motion compensation. These algorithms generally require changing existing data structures and adding function modules to the existing video decoding diagram. Furthermore, these algorithms require a substantial amount of additional computation. In contrast, there are algorithms that operate at block-level, which estimate the MV for each block. Block-level error concealment algorithms generally require much less complexity or overhead.
In one such block-level error concealment algorithm, for each 16×16 or 4×4 block, the consistency of the MVs falling inside is checked by computing the first and second order moments. However, the block-level error concealment algorithm does not provide a solution for those blocks that fail the check, which compromises performance.